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DETAILED ACTION 
Response to Amendment 

Applicant's submission filed on 04/25/2005 has been entered. Claims 1-28, 30-33, 35- 
3842-45 and 47-50 have been canceled. Claims 29, 34, 39-41 and 46 have been amended. Claims 
53-56 have been newly added. Claims 29, 34, 39-41, 46 and 53-56 are pending in the 
application. 

Response to Arguments 

Applicant's arguments with respect to claim 1 have been considered but are moot in view 
of the new ground(s) of rejection based on Bradski, G.R., "Computer Vision Face Tracking for 
Use in a Perceptual User Interface", Intel Technology Journal Q2, 1-15, (1998). 

Specification 

The disclosure is objected to because of the following informalities: for example, on line 
6 of the claim 53, "pixes" should be "pixels". On line 22 of the claim 53, "HSV array or pixels" 
should be "HSV array of pixels". On line 24 of the claim 55, "HSV array or pixels" should be 
"HSV array of pixels". Appropriate correction is required. 



Claim Objections 

Claim 53 and 55 are objected to because of the following informalities: for example, on 
line 6 of the claim 53, "pixes" should be "pixels". On line 22 of the claim 53, "HSV array or 
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pixels" should be "HSV array of pixels". On line 24 of the claim 55, "HSV array or pixels" 
should be "HSV array of pixels". Appropriate correction is required. 



Claim Rejections - 35 USC § 112 

The following is a quotation of the first paragraph of 35 U.S.C 1 12: 

The specification shall contain a written description of the invention, and of the manner and process of making 
and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it 
pertains, or with which it is most nearly connected, to make and use the same and shall set forth the best mode 
contemplated by the inventor of carrying out his invention. 

The following is a quotation of the first paragraph of 35 U.S.C. 112: 

The specification shall contain a written description of the invention, and of the manner and process of making 
and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it 
pertains, or with which it is most nearly connected, to make and use the same and shall set forth the best mode 
contemplated by the inventor of carrying out his invention. 

Claims 29, 34, 39-41, 46 and 53-56 are rejected under 35 U.S.C. 1 12, first paragraph, as 
failing to comply with the written description requirement. The claim(s) contains subject matter 
which was not described in the specification in such a way as to reasonably convey to one skilled 
in the relevant art that the inventor(s), at the time the application was filed, had possession of the 
claimed invention. 

For example, the base claim 53 recites the iterative steps (a) determining whether the 
initial test HSV array of pixels has acceptable hue, saturation, and value amounts and ranges for 
the tracking system; (b) storing the initial test HSV array of pixels in a memory if the initial test 
HSV array of pixels has the acceptable hue, saturation, and value amounts and ranges; (c) 
decrementing the established number of test windows; repeating limitations (a), (b), and (c) for 
succeeding test windows until the established number of test windows is zero, wherein any test 
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HSV array of pixels that is stored is combined with existing HSV array of pixels in the memory 
to create a combined test HSV array of pixels. The claim language is not supported by the 
specification. For example, see page 17, 18-29 and Figs. 7A-7C of applicant's specification. 
Specifically, applicant recites using "initial test HSV array of pixels" in steps (a), (b) and (c), and 
later the claim 53 further set forth repeating (a), (b) and (c) using the initial test HSV array of 
pixels or any test HSV array of pixels. If the steps (a), (b) and (c) are repeated, only the initial 
test HSV array of pixels are used, as opposed to using any combined test HSV array of pixels. 
The specification as disclosed, however, using the combined HSV array of pixels in the memory 
obtained from all of the previous iterations in order to determine whether the combined test HSV 
array of pixels has acceptable values and ranges for hue and saturation and the initial test HSV 
array of pixels are replaced . Moreover, the pixel classification map is created until all steps are 
completed, not in the intermediate steps of the iterations. Therefore, the metes and bounds of the 
coverage of at least base claim 53 (as well as the base claim 55) cannot be ascertained. 

To comply with the "written description" requirement of 35 U.S.C. 112, first paragraph, 
an applicant must convey with reasonable clarity to those skilled in the art that, as of the filling 
date sought, he or she was in possession of the invention. The invention is, for purposes of the 
"written description" inquiry, whatever is now claimed. Vas-Cath, Inc. v. Mahurkar, 935 F.2d 
1555, 1563-64, 19 USPQ2d 1111,1117 (Fed. Cir. 1991). For purposes of written description, 
one shows "possession" by descriptive means such as words, structures, figures, diagrams, and 
formulas that fully set forth the claimed invention. Lockwood v. American Airlines, Inc., 107 
F.3d 1565, 1572, 41 USPQ2d 1961, 1966 (Fed. Cir. 1997). Such descriptive means cannot be 
found in the disclosure for the inventions of the base claim 3 1 , 32 and 36. 
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Claims 29, 34, 39-41, 46 and 54 depend upon the claim 53 and are rejected due to their 
dependency on the claim 53. The claims 51-52 and 56 depend upon the base claim 55 and are 
rejected due to their dependency on the claim 55. 

Claim Rejections - 35 USC § 112 
The following is a quotation of the second paragraph of 35 U.S.C. 1 12: 

The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the 
subject matter which the applicant regards as his invention. 

Claims 29, 34, 39-41, 46 and 53-56 are rejected under 35 U.S.C. 1 12, second paragraph, 
as being indefinite for failing to particularly point out and distinctly claim the subject matter 
which applicant regards as the invention. 

For example, the base claim 53 recites the iterative steps (a) determining whether the 
initial test HSV array of pixels has acceptable hue, saturation, and value amounts and ranges for 
the tracking system; (b) storing the initial test HSV array of pixels in a memory if the initial test 
HSV array of pixels has the acceptable hue, saturation, and value amounts and ranges; (c) 
decrementing the established number of test windows; repeating limitations (a), (b), and (c) for 
succeeding test windows until the established number of test windows is zero, wherein any test 
HSV array of pixels that is stored is combined with existing HSV array of pixels in the memory 
to create a combined test HSV array of pixels. The claim language is not supported by the 
specification. For example, see page 17, 18-29 and Figs. 7A-7C of applicant's specification. 
Specifically, applicant recites using "initial test HSV array of pixels" in steps (a), (b) and (c), and 
later the claim 53 further set forth repeating (a), (b) and (c) using the initial test HSV array of 
pixels or any test HSV array of pixels. If the steps (a), (b) and (c) are repeated, only the initial 
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test HSV array of pixels are used, as opposed to using any combined test HSV array of pixels. 
The specification as disclosed, however, using the combined HSV array of pixels in the memory 
obtained from all of the previous iterations in order to determine whether the combined test HSV 
array of pixels has acceptable values and ranges for hue and saturation and the initial test HSV 
array of pixels are replaced. Moreover, the pixel classification map is created until all steps are 
completed, not in the intermediate steps of the iterations. Therefore, the metes and bounds of the 
coverage of at least base claim 53 (as well as the base claim 55) cannot be ascertained. 

Claims 29, 34, 39-41, 46 and 54 depend upon the claim 53 and are rejected due to their 
dependency on the claim 53. The claims 51-52 and 56 depend upon the base claim 55 and are 
rejected due to their dependency on the claim 55. 

Claim Rejections - 35 USC §103 

1 . The following is a quotation of 35 U.S.C 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

2. Claims 29, 34, 39-41, 46 and 53-56 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Bradski, G.R., "Computer Vision Face Tracking for Use in a Perceptual User 
Interface", Intel Technology Journal Q2, 1-15, (1998). 



3. Re Claim 53: 
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(a) Bradski teaches a method to calibrate a tracking system to determine if an image 
frame has an established number of test windows that meet hue, saturation, and value amounts or 
ranges, and then to track an object within the image frame, comprising: 

Converting the image frame from red-green-blue pixel information to a hue-saturation- 
value (HSV) array of pixels (e.g., Bradski discloses the RGB to HSV conversion in page 3 and 
13, as well as the HSV histograms are used to convert incoming video pixels to a corresponding 
probability of flesh image. Bradski discloses that the HSV brightness and saturation thresholds 
are employed with low and high thresholds are set off 10% of the maximum pixel value; see page 
1-7); 

Thresholding the HSV array of pixels to create a thresholded HSV array of pixels (e.g., 
Bradski discloses that the HSV brightness and saturation thresholds are employed with low and 
high thresholds are set off 10% of the maximum pixel value; see page 1-7); 

Setting the established number of test windows needed within the image frame for the 
calibration to be successful (e.g., Bradski teaches choosing a search window size and the initial 
location of the search window including the initial test window and subsequently computes the 
mean location of the search window or computes the search window size as a function of the 
zero-th moment; see page 5 for the Coupled CAMSHIFT Algorithm of Bradski wherein a set 
number of iterations are performed for the calibration. Bradski discloses in Page 4 convergence 
occurs in six iterations for the actual sub-sampled flesh color probability distribution of an 
image of a face and a nearby hand); 

Establishing an initial test window in the thresholded HSV array of pixels to create an 
initial test HSV array of pixels (e.g., Bradski teaches choosing a search window size and the 
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initial location of the search window including the initial test window and subsequently 
computes the mean location of the search window or computes the search window size as a 
function of thezero-th moment; see page 5 for the Coupled CAMSHIFT Algorithm of Bradski)] 

(a) Determining whether the initial test HSV array of pixels has acceptable hue, 
saturation, and value amounts and ranges for the tracking system (e.g., determining the HSV 
histograms or the flesh probability image with the thresholding values also determines the HSV 
array of pixels having acceptable hue, saturation, and value amounts and ranges for the tracking 
system to track the human face; see Page 3-7 and 9-1 3); 

(b) Storing the initial test HSV array of pixels in a memory if the initial test HSV array of 
pixels has the acceptable hue, saturation, and value amounts and ranges (e.g., Bradski discloses 
in page 3 storing flesh color histogram as a lookup table and thereby the color HSV array of 
pixels are stored in a memory)] 

(c) decrementing the established number of test windows (e.g., Bradski discloses 
repeating the iterative steps in the Mean Shift Algorithm, in Continuously Adaptive Mean Shift 
Algorithm and in the Coupled CAMSHIFT Algorithm of Pages 5-5 until convergence or a set 
number of iterations and thereby the number of iterations is decremented, at each iteration, a 
test window is constructed and thereby the number of the test windows is decremented); 

Repeating limitations (a), (b) and (c ) for succeeding test windows until the established 
number of test windows is zero (e.g., Bradski discloses repeating the iterative steps in the Mean 
Shift Algorithm, in Continuously Adaptive Mean Shift Algorithm and in the Coupled CAMSHIFT 
Algorithm of Pages 3-5 until convergence or a set number of iterations and thereby the number 
of iterations is decremented, at each iteration, a test window is constructed and thereby the 
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number of the test windows is decremented), wherein any test HSV array of pixels that is stored 
is combined with existing HSV array of pixels in the memory to create a combined test HSV 
array of pixels (e.g., Bradski discloses calculating the centroid of the 2D color probability 
distribution within its 2D window of calculation, re-centering the window, then calculating the 
area for the next window size. The calculation of the distribution is restricted to a smaller image 
region surrounding the current CAMSHIFT window; see Page 5. Thus, Bradski implicitly 
discloses combining the existing HSV histogram in the memory to find the center and the size of 
the search window and thus to find or create a combined test HSV array of pixels); and 

Determining whether the combined test HSV array of pixels has acceptable values and 
ranges for hue and saturation (e.g., determining the HSV histograms or the flesh probability 
image with the thresholding values also determines the HSV array of pixels having acceptable 
hue, saturation, and value amounts and ranges for the tracking system to track the human face; 
see Page 3-7and 9-13); and creating a pixel classification map for the image frame if the 
combined test HSV array of pixels has acceptable values and ranges for hue and saturation (e.g., 
Bradski discloses calculating the centroid of the 2D color probability distribution within its 2D 
window of calculation, re-centering the window, then calculating the area for the next window 
size. The calculation of the distribution is restricted to a smaller image region surrounding the 
current CAMSHIFT window; see Page 5. Thus, Bradski implicitly discloses combining the 
existing HSV histogram in the memory to find the center and the size of the search window and 
thus to find or create a combined test HSV array of pixels. Finally, Bradski discloses a objects 
are tracked in real time and meanwhile a HSV image is formed after the convergence in which 
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the algorithm has been modified to adapt dynamically to the probability distribution of the object 
it is tracking; see Page 3-7 and 9-13). 

(b) Although Bradski is silent to the term" combined test HSV array of pixels" or "pixel 
classification map", Bradski discloses calculating the centroid of the 2D color probability 
distribution within its 2D window of calculation, re-centering the window, then calculating the 
area for the next window size. The calculation of the distribution is restricted to a smaller image 
region surrounding the current CAMSHIFT window; see Page 5. The calculation involves the 
mean shift locations and window size derived from the sequence of iterations in which the each 
test window at each iteration is involved in the calculation of the location and size of the new test 
window. Thus, Bradski implicitly discloses combining the existing HSV histograms in the 
memory to find the center and the size of the new search window and thus to find or create a 
combined test HSV array of pixels. As to the term "pixel classification map", Bradski discloses 
that objects are tracked in real time and meanwhile a HSV image is formed after the convergence 
in which the algorithm has been modified to adapt dynamically to the probability distribution of 
the object it is tracking; see Page 3-7 and 9-13. Therefore, the object is dynamically tracked 
using the dynamically adjusted test window wherein the color histogram lookup in the 
calculation regions is formed and thus the pixel classification map is established. 

(d) It would have been obvious to one of ordinary skill in the art to have used "the 
combined test HSV array of pixels" because this combination would have allowed the size and 
location of the probability distribution changes in time as the tracked object moves in a video 
sequences. ", Moreover, Bradski suggests the claim limitation for the reasons set forth below. 
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Bradski discloses calculating the centroid of the 2D color probability distribution within its 2D 
window of calculation, re-centering the window, then calculating the area for the next window 
size. The calculation of the distribution is restricted to a smaller image region surrounding the 
current CAMSHIFT window; see Page 5. The calculation involves the mean shift locations and 
window size derived from the sequence of iterations in which the each test window at each 
iteration is involved in the calculation of the location and size of the new test window. Thus, 
Bradski implicitly discloses combining the existing HSV histograms in the memory to find the 
center and the size of the new search window and thus to find or create a combined test HSV 
array of pixels based on the existing set of HSV histograms. 

(e) One having the ordinary skill in the art would have been motivated to do this because 
it would have accurately tracked an object as it moves based on the history of the tracking record 
wherein the moment of the probability distribution is used based on the combined test HSV 
histograms (See Page 4-5 of Bradski). 

Claim 29, 34, 39-40, 54: 

The claims further set forth the claim limitation of "determining a mean saturation of the 
initial test HSV array of pixels; determining if the mean saturation of the initial test HSV array of 
pixels falls within a first predetermined range; determining if a standard deviation of saturation 
of the initial test HSV array of pixels is less than a first predetermined amount if the mean 
saturation of initial test HSV array of pixels falls within the first predetermined range; 
determining a mean hue of the initial test HSV array of pixels if the standard deviation of the 
saturation of the initial test HSV array of pixels is less than the first predetermined amount and 
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determining a standard deviation of hue of the initial test HSV array of pixels if the mean hue of 
the initial HSV array of pixels falls within a second predetermined range". 

However, Bradski discloses calculating the mean and standard deviation of the initial test 
HSV array of pixels. See Page 3-7 and 9-13. Bradski discloses HSV brightness and saturation 
thresholds are employed since hue is not well defined for very low or high brightness or low 
saturation and low and high thresholds are set off 10% of the maximum pixel value; Page 7. 
Bradski discloses adjusting the nature of the probability distribution by properly setting the HSV 
brightness and saturation thresholds. 

Bradski is silent on the claim limitation of "determining a mean saturation of the initial 
test HSV array of pixels; determining if the mean saturation of the initial test HSV array of 
pixels falls within a first predetermined range; determining if a standard deviation of saturation 
of the initial test HSV array of pixels is less than a first predetermined amount if the mean 
saturation of initial test HSV array of pixels falls within the first predetermined range" and 

"determining a mean hue of the initial test HSV array of pixels if the standard deviation 
of the saturation of the initial test HSV array of pixels is less than the first predetermined 
amount, and determining a standard deviation of hue of the initial test HSV array of pixels if the 
mean hue of the initial HSV array of pixels falls within a second predetermined range." 

(c) However, Bradski teaches a standard mean shift method to track objects that have 
been converted into probability distributions to determine if the search was moved by a value 
less than a preset threshould value . Bradski teaches determining window sizing parameters for a 
set of search windows or boxes or windows of displaying images, each of the moving search 
windows having a same shape and a same pixel size as the calibration region in a form of 
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rectangle or a search window, wherein tracking data, to track the selected gestures is selected 
from one of the calculation region and the search windows having a highest tracking probability . 
For example, Bradski teaches that the tracking data such as window location parameters are 
determined/adjusted and a search window having the largest connected region of a probability 
distribution and the greatest probability density is selected and each of the adjacent test windows 
share at least one pixel with the calibration rectangle (e.g., Bradski Page 3-7 and 9-13). 

Bradski further discloses a HSV probability distribution which involves the mean hue or 
the standard deviation of the hue and search for the area with the greatest probability density 
involves the motion-tracking processing suggesting the mean hue or the standard deviation of the 
hue being less than predetermined levels wherein the small mean saturation and a small standard 
deviation of a saturation or hue of the pixels in a Gaussian or Chi-Square probability density 
results in the optimum tracking accuracy (Bradski Pages 3-7 and 9-13). Therefore, Bradski 
teaches the claim limitation relating to the selection of a test HSV array of pixels by comparing 
the hue and saturation zero-th, 1 st and 2 nd moments with the predetermined parameter values. 

(d) It would have been obvious to one of ordinary skill in the art to have used the 
optimum ranges/thresholds for the mean and standard deviation of the hue or saturation to obtain 
the optimum tracking accuracy or minimum tracking errors because Bradski teaches performing 
pattern recognition based on HSV histograms or probability distributions to determine a best 
match for the tracked object because such a selection of optimum ranges yields the best match 
tracking object based on the probability distribution or probability density. 

(e) One having the ordinary skill in the art would have been motivated to do this because 
it would have provided a routine experimentation with the optimum ranges and criteria based on 
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some specific parameters of the probability distributions to obtain the greatest probability density 
for object selection to find the best match tracking object. See In re Peterson, 65 USPQ2d 1379 
(CA FC 2003) and In re Geisler (CA FC) 43 USPQ2d 1362. 



Re Claims 55, 41, 46, 51-52, and 56: 

(a) The claims 55, 41, 46, 51-52, and 56 encompass the same scope of invention as that 
of claims 53, 2934, 39-40 and 54 except additional claimed limitation of "a machine-readable 
medium having recorded thereon instructions". 

(b) Bradski has taught a method of calibrating a computer- vision system to track a 
selected object through a series of frames of data. 

(c) Bradski however does not particularly disclose a recording medium having recorded 
thereon instructions. 

(d) However, one of ordinary skill in the art would have recognized that computer 
readable medium (i.e., floppy, cd-rom, etc.) carrying computer-executable instructions for 
implementing a method, because it would facilitate the transporting and installing of the method 
on other systems, is generally well-known in the art. For example, a copy of the Microsoft 
Windows operating system can be found on a cd-rom from which Windows can be installed onto 
other systems, which is a lot easier than running a long cable or hand typing the software onto 
another system. The Office takes Official Notice of this teaching. 
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(e) Therefore, it would have been obvious to implement the Bradski's method and put 
Bradski's program on a computer readable medium, because it would facilitate the transporting, 
installing and implementing of Bradski's program on other systems. 



Conclusion 

Applicant's amendment necessitated the new ground(s) of rejection presented in this 
Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). 
Applicant is reminded of the extension of time policy as set forth in 37 CFR 1 .136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1 .136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the date of this 
final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Jin-Cheng Wang whose telephone number is (571) 272-7665. 
The examiner can normally be reached on 8:00 - 6:30 (Mon-Thu). 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Mike Razavi can be reached on (571) 272-7664. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 



jew 




